System and method for statistically separating and characterizing noise which is added to a signal of a machine or a system

ABSTRACT

Method for finding the probability density function type and the variance properties of the noise component N of a raw signal S of a machine or a system, said raw signal S being combined of a pure signal component P and said noise component N, the method comprising: (a) defining a window within said raw signal; (b) recording the raw signal S; (c) numerically differentiating the raw signal S within the range of said window at least a number of times m to obtain an m order differentiated signal; (d) finding a histogram that best fits the m order differentiated signal; (e) finding a probability density function type that fits the distribution of the histogram; (f) determining the variance of the histogram, said histogram variance being essentially the m order variance σ 2   (m)  of the noise component N; and (g) knowing the histogram distribution type, and the m order variance σ 2   (m)  of the histogram, transforming the m order variance σ 2   (m)  to the zero order variance σ 2   (0) , σ 2   (0)  being the variance of the pdf of the noise component N, and wherein the histogram type as found in step (e) being the probability density function type of the noise component N.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/109,061, filed Apr. 18, 2005, which claims foreign priority benefitsfrom Israeli Patent Application No. 166837, filed Feb. 10, 2005. Theentire content of both applications are each incorporated by referenceherein.

FIELD OF THE INVENTION

The present invention relates to the field of estimating andcharacterizing noise which is added to a signal of a machine or asystem. More particularly, the invention relates to a method and systemfor estimating and characterizing the component of the added noise of asignal. The method of the invention enables the finding of both thestatistical nature and the type of the probability distribution ordensity functions of the noise component as well as its variance.

BACKGROUND OF THE INVENTION

The importance of the knowledge of the fundamental properties ofstochastic systems and processes has been recently acknowledged by agrowing portion of the scientific and engineering community. Among otherproperties of stochastic processes, the nature of the noise componentwhich contaminates the pure signal of the system is of major importance.The term “noisy signal” or “raw signal” whenever referred to in thisapplication, refers to a signal which comprises a noise component and apure signal which are inseparable. Throughout this application, the term“noise” refers to any random or unknown component whose exact behaviorcannot be exactly predicted, but knowing its probability densityfunction is highly valuable. Also, the term “variance” relates to thesecond moment of the probability density function and is used as iscommon in the art of Statistics and Probability theories. Moreover,throughout this application the terms “machine”, “system” and “process”are used interchangeably with respect to the method of the invention. Anaccurate estimation of the noise properties can provide to the systemdesigner very important tools for improving the system behavior. Anaccurate determination of the noise properties is particularly importantfor dynamical systems where non-linear behavior is expected and in whichthe noise may seriously alter any estimation of the states of thesystem, if not to cause a total divergence of the parameters of thesystem model. Such conditions are particularly common in non-linearsystems when modeled by recursive or adaptive methods such as Weiner orKalman filtering. The principles and theory of Kalman and Weinerfiltering are described, for example, in Gelb, A., “Applied OptimalEstimation”, Chapter 1, pp. 1-7, The MIT Press, Cambridge, Mass., 1974.

The following United States patents are believed to represent the stateof the art for Signal estimation, noise characteristics, and Kalman andadaptive filtering in applicable systems: U.S. Pat. Nos. 6,829,534;6,740,518; 6,718,259; 6,658,261; 6,836,679; 6,754,293; and 6,697,492.

The theory of non-linear filtering and its applications are discussedin:

-   (a) Grewal, M. S. et al., Kalman Filtering, Prentice-Hall, 1993;-   (b) Jazwinski, A. H., Stochastic Processes and Filtering Theory,    Academic Press, New York, 1970, chapters 1 and 2, pp. 1-13;-   (c) Gelb, A., Applied Optimal Estimation, The MIT Press, Cambridge,    Mass., 1974 Chapter 1, pp. 1-7; and-   (d) Wiener, N., Journal of Mathematical and Physical Sciences 2, 132    (1923).

The art of signal processing, probability and stochastic processes andnoise characteristics are also discussed in:

-   (a) Bruno Aiazzi et al., IEEE Signal Processing Lett. 6 138 (1999);-   (b) R. Chandramouli et al., “Probability, Random Variables and    Stochastic Processes”, A. Papoulis, McGraw-Hill USA, (1965);-   (c) IEEE Signal Processing Lett. 6 129;-   (d) Zbyszek P. Karkuszewski, Christopher Jarzynski, and Wojciech H.    Zurek, Phys Rev. Lett. 89, 170405 (2002);-   (e) A. F. Faruqi and K. J. Turner Applied Mathematics and    Computation, 115, 213 (2000);′-   (f) J. P. M. Heald and J. Stark, Phys. Rev. Lett. 84, 2366 (2000);-   (g) A. A. Dorogovtsev, Stochastic Analysis and Random Maps in    Hilbert Space, VSP Publishing, The Netherlands, (1994) (in    particular see the consideration for high-order stochastic    derivative in chap. 1);-   (h) H. Kleinert and S. V. Shabanov, Phys. Lett. A, 235, 105, (1997);-   (i) Elachi, C., Science, 209, 1073-1082, (1980);-   (j) Valeri Kontorovich et al., IEEE Signal Processing Lett. 3, 19    (1996);-   (k) Steve Kay., IEEE Signal Processing Lett. 5, 318 (1998);-   (l) Michael I. Tribelsky, Phys. Rev. Lett. 89, 070201 (2002).

The theory of curve fitting, differentiation and high order derivativesis discussed in:

-   (a) G. Di Nunno, Pure Mathematics 12, 1, (2001); and-   (b) K. Weierstrass, Mathematische Werke, Bd. III, Berlin 1903, pp.    1-17.

It is an object of the present invention to provide a method for thestatistical separation and determination of the noise properties fromthe noisy signal.

It is another object of the invention to provide such a method that canbe performed in real-time.

It is still another object of the present invention to provide such amethod for characterizing the noise which is adaptive.

It is still another object of the present invention to provide saidmethod for characterizing the noise that can determine not only thevariance of the noisy signal, but also the type of the probabilitydensity function (pd) of the noise component.

It is still another object of the present invention to provide saidmethod for characterizing the noise that does not depend on a prioriknowledge of the structure of the pure signal.

It is still another object of the present invention to provide saidmethod for characterizing the noise that does not depend on thestructure of the pure signal.

It is still another object of the present invention to provide saidmethod for characterizing the noise that involves defining a window ofthe analyzed signal, and given said window, the method does not dependon any accumulative information outside said window boundaries.

Other objects and advantages of the present invention will becomeapparent as the description proceeds.

SUMMARY OF THE INVENTION

The present invention refers to a method for finding the probabilitydensity function and the variance properties of the noise component N ofa raw signal S of a machine or a system, said raw signal S beingcombined of a pure signal component P and said noise component N, themethod comprising the steps of: (a) defining a window within said rawsignal; (b) recording the raw signal S; (c) numerically differentiatingthe raw signal S within the range of said window at least a number oftimes m to obtain an m order differentiated signal; (d) finding ahistogram that best fits the m order differentiated signal; (e) findinga probability density function type that fits the distribution of thehistogram; (f) determining the variance (or any equivalent parameter,depending on the specific said pdf type) of the histogram, saidhistogram variance being essentially the m order variance σ² _((m)) ofthe noise component N; and (g) knowing the histogram distribution type,and the m order variance or σ² _((m)) of the histogram, transforming them order variance σ² _((m)) to the zero order variance σ² ₍₀₎, said σ²₍₀₎ being the variance of the pdf of the noise component N, and whereinthe histogram type as found in step (e) being the probability densityfunction type of the noise component N.

Preferably, the method is repeatedly performed as the raw signal Sprogresses.

Preferably, the method is performed in real-time.

Preferably, the probability density function type that fits thedistribution of the histogram is the one from a list that found to bebest fitting the distribution of the histogram.

Preferably, the list comprises only one probability density functiontype.

Preferably, the one probability density function type is the Gaussiantype.

Preferably, the transformation is performed by means of a specificexpression suitable for the said fitted probability density function,wherein said specific expression is derived from the following generalexpression

$F_{n_{i}}^{(m)} = {\int_{D_{z}}{\int{\left\{ {\prod\limits_{j = 1}^{m}{S_{j}^{m}{f(\xi)}}} \right\} {\xi^{(m)}}}}}$

Preferably, when the fitted probability distribution function isGaussian, the transform is performed by means of the following specificexpression

$\frac{^{m}{N\left( {0,\sigma_{0}^{2}} \right)}}{^{m}} = {{\alpha (m)}{N\left( {0,{{\beta (m)}\sigma_{0}^{2}}} \right)}}$

The present invention also relates to an apparatus for determining theprobability density function type and the variance properties of thenoise component N of a raw signal S of a machine or a system, said rawsignal S being combined of a pure signal component P and said noisecomponent N, the system comprises: (a) differentiating module, forreceiving and numerically differentiating the raw signal S within therange of a predefined window at least a number of times m to obtain an morder differentiated signal; (b) a module for finding a histogram thatbest fits the m order differentiated signal; (c) a list containing atleast one type of predefined probability density function; (d) a modulefor finding one probability density function type from said list thatbest fits the distribution of the histogram; (e) a module fordetermining the variance of the histogram, said histogram variance beingessentially the m order variance σ² _((m)) of the noise component N; and(f) a module for, given the histogram distribution type and the m ordervariance σ² _((m)) of the histogram, transforming the m order varianceσ² _((m)) to the zero order variance σ² ₍₀₎, said σ² ₍₀₎ being thevariance of the pdf of the noise component N, wherein the histogram typeas found in step (d) being the probability density function type of thenoise component N.

Preferably, the apparatus components operate repeatedly to find theupdated probability density function type and the variance properties ofthe noise component as the signal S progresses.

The present invention also relates to a system for receiving a rawsignal S which is combined from a pure signal P and a noise component N,and for outputting a signal which is essentially said pure signal,wherein the system comprises: (a) apparatus as described above forreceiving said raw signal and outputting the probability densityfunction and distribution type of said noise component into a filter;and (b) a filter receiving said raw signal and also receiving saidprobability density function and distribution type of said noisecomponent from said apparatus, and given said received data, processingand outputting a signal which is essentially said pure signal.

Preferably, the filter is an adaptive filter.

Preferably, the filter is a Kalman Filter.

Preferably, the system of the invention as described above operatescontinuously in real time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a general system according to the prior art.

FIG. 2 illustrates a simulated raw signal S that contains a purecomponent P and a noise component N.

FIG. 3 shows the 50^(th) differentiation of signal S of FIG. 2.

FIG. 4 shows a histogram that was drawn for the 50 times differentiatedsignal of FIG. 3. The solid line in FIG. 4 represents a Least MeanSquare fit of the point of the histogram to a Gaussian Functions fromwhich the Gaussian parameters are extracted.

FIG. 5 shows the re-normalized values σ₍₀₎ as extracted from thecorresponding σ_((m)) found for the raw signal of FIG. 2 that wasdifferentiated m=1, 2, 3, . . . , 200 times.

FIG. 6 illustrates how the method of the present invention can be usedin conjunction with a Kalman filter.

FIG. 7 illustrates a simulation of the first derivative (m=1) of a 200Knormal distributed, N(0, σ₀ ²≦1), noise signal (only partially shown)with the corresponding deduced histogram (fitted to a Gaussian andarbitrarily scaled). Also shown is the original density function used togenerate the noise signal (dashed curve). As expected, σ=√{square rootover (2)}σ₀.

FIG. 8 illustrates a simulation of the Fifth derivative (m=5) of anormal distributed, noise signal (see also FIG. 7). As expected,σ=√{square root over (252)}σ₀.

FIG. 9 illustrates an apparatus for performing a method according to oneembodiment of the invention.

DETAILED DESCRIPTION

As said, the knowledge of the properties of the expected noise componentprovides to the system designer a very significant tool for improvingthe system structure and behavior. The essence of this invention is toperform relatively simple numerical calculations on the noisy signal inorder to derive both the type of the probability density function of thenoise and the properties of said of the probability density function andin particular the variance of said function.

FIG. 1 shows a typical system. A raw signal S which is combined of apure signal P and an additive noise N component is provided to thesystem. Throughout this application, it should be noted that theinvention relates to the characterizing of the noise which is added notonly to an input signal, but may also relate to a noise that is added toa signal within the system. The present invention can provide theproperties of the noise component N, given said signal S.

The method of the present invention comprises of the following steps:

-   1. Defining a portion of the raw signal that may dynamically    progress according to the development of the signal, hereinafter    defined as the “window”, or the “analysis window”, provided that the    number of elements in said window is statistically sufficient;-   2. Recording the noisy signal S which is combined of a pure signal    portion P and of a noise component N;-   3. Numerically differentiating the raw signal at least a number of    times m to obtain an m times differentiated signal;-   4. Finding the histogram of the differentiated signal;-   5. Providing a list of optional probability density functions, and    from said list finding the one probability density function that is    best fitted to the histogram distribution;-   6. Determining from the best fitted probability density function the    parameters that characterize that function, said function being the    probability density function of the m order differentiated raw    signal S, but essentially being a close approximation to the in    order differentiated probability density function of the noise    component N. The arguments for supporting this assumption are given    hereinafter;-   7. Transforming the parameters of the fitted, in order    differentiated probability density function to extract the    parameters of the zero order probability density function of the    noise component N of signal S. The transformation is performed using    an expression which is suitable for the fitted probability density    function type (as will be elaborated later, expressions (3) and (4)    which are given below are general expressions that are suitable for    any type of probability density function, while the simplified    expression (5) is suitable for Gaussian probability density    function);

FIG. 9 shows an apparatus for performing a method according to oneembodiment of the invention. In particular, FIG. 9 shows a NoiseEstimation Unit 12 and provides a block diagram illustrating the method.A noisy, raw signal S=P+N which is combined of a pure component P and anoise component N is provided over line 40 (step 2 above) to a NoiseEstimation Unit 12. The signal S is differentiated m times by thedifferentiation block 41 (step 3 above). Then, block 42 draws ahistogram for the (m) times differentiated signal as provided by block41 (step 4 above). Block 43 receives the histogram from block 42, andfinds a probability density function type that best fits thedistribution of the histogram (step 5 above). For that purpose, block 43may use the library 44 containing several probability density functiontypes to find the one probability density function that best fits thehistogram, or alternatively it may apply an assumed probability densityfunction from block 45 (for example, a Gaussian distribution) (also step5 above). After finding the probability density function that best fitsthe histogram, block 47 which receives the probability density functiontype over line 50, and the histogram over line 51 determines the (m)order variance σ² _((m)) of the histogram (step 6). The (m) ordervariance, as well as the probability density function type are providedinto the transformation block 46, which in turn uses this data (pdf)type and σ² _((m)) in order to find the zero order variance σ² ₍₀₎ whilethe pdf type remains the same as for the m order pdf (step 7). Block 46then outputs both the zero order probability density function type andthe variance σ² ₍₀₎ of the noise to any system that may use thesevaluable parameters of the noise.

Now, the present invention will be described by means of an example.FIGS. 2 to 5 demonstrate the method of the present invention.

Example

FIG. 2 illustrates a simulated raw signal S that contains a purecomponent P and a noise component N. The duration of the S signal (i.e.,the “window” considered) was of 0.7 s. It should be clear to any one whois skilled in the art that the window's length can be shorter or longer,depending on the specific case considered. It should also be clear toany one who is skilled in the art that the length of the window can beequal to or, preferably, shorter than the length of the signal. Thenoise component was intentionally selected to have Gaussian probabilitydensity function with a σ₍₀₎=14. The S signal of FIG. 2 was numericallydifferentiated 200 times (step 3 above). The 50^(th) differentiation ofsignal S is shown in FIG. 3. From the differentiation result of FIG. 3,a histogram was drawn as shown in FIG. 4 (the discrete points form thishistogram). A Gaussian function was then fitted (in the Least MeansSquare sense) to yield the solid line of FIG. 4. Then, the parameters ofsaid 50^(th) order differentiation probability density function of FIG.4 were extracted. More particularly, the σ₍₅₀₎ was found to be4.432413969422223e+015. Next, using and an expression σ₀=ƒ(σ_(m)) (i.e.,the initial value is a function of the extracted value after thedifferentiation step) that will be elaborated further hereinafter, σ₍₀₎was found to be 13.958. In addition, from the same expression and thevarious σ_((m)), the value of σ₍₀₎ was separately extracted. FIG. 5shows the extracted values of σ₍₀₎ as found according to the method ofthe present invention for the raw signal of FIG. 2 that wasdifferentiated m=1, 2, 3, . . . , 200 times. It can be seen that σ₍₀₎was found to be very close to the intended, initial value of 14 for allsaid values of m. More particularly, the σ₍₀₎ of the noise component wasfound to be very close to the value the intended, initial value of 14 aswas pre-selected for the pdf (probability density function) of the noisecomponent N in this simulation. It can also be concluded that m of aslow as 4 or 5 may be sufficient to extract the value of σ₍₀₎ with highaccuracy, as the calculated σ₍₀₎ for all m larger than 5 are extremelyclose to the original value 14. Therefore, it can also be concluded thatin most cases there is no practical need to differentiate to ordershigher than 10.

One of the advantages of the invention as described is the fact that themethod can be relatively easily performed in real-time, as the amount ofdata that is necessary for performing the analysis is relatively small,i.e., only to the extent of statistical validity. Moreover, the methodrequires the use of very limited amount of memory resources, as nohistorical data of the signal is advantageous. The only informationnecessary is that contained in the selected window, and the window inmost cases can be narrow.

The present invention is applicable to most types of probability densityfunctions. For each type of pdf one can easily derive the suitableexpression as is necessary in step 7 above. Therefore, it is preferablyrecommended to keep in the list of step 5 above at least one type ofprobability density function, or preferably more, to keep thosefunctions that are most expected for noise probability densityfunctions.

Theoretical Considerations

Considering a stochastic process ξ(n_(i)), with n_(i) the collection ofstochastic events, in a measurable space (state-space) so that variancevalues of the stochastic variables considered here are finite, adifferentiating operator, operating on a signal vector, may be definedwith respect to the index of the signal data points in their sequencedorder (or equivalently, treating the signal as a time series vector witha unit time step). By doing this, one may realize that a differentiationprocedure, of the first order, is equivalent to numerical subtractingthe element n_(i) from the element n_(i+1), in the stochastic signal.Since in such random set of points each point is totally independent ofall other points and correlated to any other data point within the setonly by the mutual statistics of the sample space, denoted by Ω (i.e.all points (i, j) are uncorrelated where i≠′j), the equivalence tosubtracting the element n_(i), from the element n_(i+1) in the noisesignal would be the equivalent of the subtraction of two independentRandom Variables with identical statistical distribution (IID).

In contrast with the case of the first derivative, where one couldassume that all individual data points were uncorrelated, higher orderderivatives involve correlated expressions that lead, in the generalcase, to non-trivial expressions for the resultant probabilityfunctions.

Considering the above definitions and referring to some arbitrary randomvariable function V(n_(i), ξ), referred here as the original data signalwith ξ as the stochastic random variable, one can now derive the secondorder derivative index series, V⁽²⁾(n_(i) ⁽²⁾,ξ⁽²⁾), with ξ⁽²⁾ refers tothe (yet) unknown stochastic random variable corresponding to thesecond-order derivative vector by realizing that n_(i) ⁽¹⁾=n₁−n_(i−1)and n_(i+1) ⁽¹⁾=n_(i+1)−n₁ so that n_(i) ⁽²⁾=n_(i+1) ⁽¹⁾−n_(i)¹=n_(i+1)−2n_(i)+n_(i−1). These expressions imply that the probabilitydensity function of the second order derivative is the equivalent pdf ofthe sum of three independent, however non-identical, random variables(InID), all with similar, however not identical, probability densityfunctions. Referring now to the general result that given twoindependent random variables ξ₁ and ξ₂ on the space R^(k) with μ and νtheir respective distribution functions and ƒ and g denote theirrespective density functions, than the distribution of the sum ξ₁+ξ₂ isthe convolution μ*ν and the analogue density function of the sum equalsthe convolution integral denoted by ƒ*g.

Using the notation ƒ_(ξ) ₁ _(ξ) ₂ _(ξ) ₃ =ƒ_(ξ) ₁ *ƒ_(ξ) ₂ *ƒ_(ξ) ₃ thisimplies:

ƒ_(n) _(i) ₍₂₎=(ƒ_(n) _(i) *ƒ_((−2n) _(i) ₎)*f_(n) _(i)   (1)

Following the above arguments, for higher derivatives, it can now easilybe deduced that the m'th derivative of a random variable derived from anarbitrary statistically defined variable can be obtained by noting thatthe correlation elements that dictate the derivative expressions aregiven by the matrix (hereinafter: “the Stochastic-Derivative Matrix”):

$\begin{matrix}\; & \; & \; & \; & \; & \; & \; & \; & 1 & \; & \; & \; & \; & \; & \; & \; & \; \\\; & \; & \; & \; & \; & \; & \; & 1 & \; & {- 1} & \; & \; & \; & \; & \; & \; & \; \\\; & \; & \; & \; & \; & \; & 1 & \; & {- 2} & \; & 1 & \; & \; & \; & \; & \; & \; \\\; & \; & \; & \; & \; & 1 & \; & {- 3} & \; & 3 & \; & {- 1} & \; & \; & \; & \; & \; \\\; & \; & \; & \; & 1 & \; & {- 4} & \; & 6 & \; & {- 4} & \; & 1 & \; & \; & \; & \; \\\; & \; & \; & 1 & \; & {- 5} & \; & 10 & \; & {- 10} & \; & 5 & \; & {- 1} & \; & \; & \; \\\; & \; & 1 & \; & {- 6} & \; & 15 & \; & {- 20} & \; & 15 & \; & {- 6} & \; & 1 & \; & \; \\\; & 1 & \; & {- 7} & \; & 21 & \; & {- 35} & \; & 35 & \; & {- 21} & \; & 7 & \; & {- 1} & \; \\1 & \; & {- 8} & \; & 28 & \; & {- 56} & \; & 70 & \; & {- 56} & \; & 28 & \; & {- 8} & \; & 1\end{matrix}$         …

and are governed by the following expression, denoted here as theStochastic-Derivative matrix S_(k) ^(m), and given by

$\mspace{11mu} {{S_{j}^{(m)} = {\left( {- 1} \right)^{j + 1}\begin{pmatrix}m \\j\end{pmatrix}}},}$

where

$\quad\begin{pmatrix}m \\j\end{pmatrix}$

denote the elements of the binomial coefficients, and theStochastic-Derivative matrix S_(k) ^(m), as defined above is in fact avariant of Pascal Triangle.

In terms of a summation of the individual elements needed to account forthe probability density function of the m'th order numerical derivative,the summation may be written as:

$\begin{matrix}{f_{m} = {\sum\limits_{j = 1}^{m}\; {S_{j}^{(m)}{f(z)}}}} & (2)\end{matrix}$

wherein ƒ(z) represents the probability density function of the originalrandom variable. For instance, for the second derivative this isequivalent to ƒ_(n) _(i) ₍₂₎=(ƒ_(n) _(i) *ƒ_((−2n) _(i) ₎)*ƒ_(n) _(i) .Generalizing the above, for a set of random variables ξ₁, ξ_(m) and afunction z=g(ξ₁, ξ₂, . . . ξ_(m)), one can form a new random variable:ξ_(z)=g(ξ₁, ξ₂, . . . ξ_(m)). In particular, the density anddistribution functions of ξ_(z), in terms of the density anddistribution functions of ξ₁, ξ₂, . . . ξ_(m) can easily be obtained. Todo so one denotes D_(z)={(ξ₁, ξ₂, . . . ξ_(m)):g(ξ₁, ξ₂, . . . ξ_(m))≦z}noting that (ξ_(z)≦z)={g(ξ₁, ξ₂, . . . ξ_(m))≦z}={(ξ₁, ξ₂, . . .ξ_(m))εD_(z)} so that:

F _(z)(z)=P(Z≦z)=P((ξ₁, ξ₂, . . . ξ_(m))εD _(z))

which gives:

 F_(z)(z) = ∫_(D_(z)) ∫f_(ξ₁, ξ₂, …ξ_(m))(ξ₁, ξ₂, …  ξ_(m))ξ₁, ξ₂, …  ξ_(m)

Thus, in order to find the distribution probability function of the newrandom variable ξ_(z), given the distribution functions of the randomvariables ξ_(j)'s, one needs to define the range of the validity of thenew variable z and to evaluate the integral using the mutual densityfunction.

For the case of independent random variables, the above expressionsimplifies with the integrand replaced by

$\; {\prod\limits_{j = 1}^{m}\; {f_{\xi_{j}}.}}$

Finally:

$\begin{matrix}\begin{matrix}{{F_{z}(z)} = {\int_{D_{z}}^{\;}{\int{\prod\limits_{j = 1}^{m}\; {f_{\xi_{j}}{\xi_{1}}\ {\xi_{2}}\mspace{14mu} \ldots \mspace{14mu} {\xi_{N}}}}}}} \\{= {\int_{D_{z}}^{\;}{\int{\left\lbrack {\prod\limits_{j = 1}^{m}{S_{j}^{m}{f_{j}(\xi)}}} \right\rbrack {\xi^{(m)}}}}}}\end{matrix} & (3)\end{matrix}$

Since the density function is the same for all individual elements ofthe multiplication term under the integral, expression (3) cansymbolically be written as:

$\begin{matrix}{{F_{n_{i}}^{(m)} = {\int_{D_{z}}^{\;}{\int{\left\{ {\prod\limits_{j = 1}^{m}{S_{j}^{m}{f(\xi)}}} \right\} {\xi^{(m)}}}}}},} & (4)\end{matrix}$

wherein F_(n) _(i) ^((m)) represents the probability distributionfunction of

$\; \frac{\partial^{m}{V\left( {n_{i},\xi} \right)}}{\partial ^{m}}$

that can be easily evaluated to derive the respective density function,recalling that the term

$\; {\prod\limits_{j = 1}^{m}{S_{j}^{m}{f(\xi)}}}$

really represents a convolution of the original probability functionweighted accordingly.

The following discussion is focused on the case where the probabilitydensity function of the noise statistics is Gaussian. For the Gaussiancase, the analysis yields a relatively straightforward expression as theGaussian pdf belongs to the few probability functions that convolve intosimilar functions. A Gaussian distribution is therefore considered,where ξ is referred to as the random variable, N(0,σ₀ ²), i.e. aGaussian distribution with the first moment equals zero, and thevariance is given by σ₀ ² as an illustrative probability (the derivationof the following with mean values other than zero is straightforward).

For the above, it can be found that the following expression (5)explicitly describes the resultant statistics, wherein β(m) is the sumof the squares of the elements of the m+1's row in theStochastic-Derivative matrix given above, and α(m) is the inverse of thesquare-root of the sum of the squares of the elements of the m+1 's rowof the Stochastic-Derivative matrix given above.

$\begin{matrix}{\frac{^{m}{N\left( {0,\sigma_{0}^{2}} \right)}}{^{m}} = {{\alpha (m)}{N\left( {0,{{\beta (m)}\sigma_{0}^{2}}} \right)}}} & (5)\end{matrix}$

Note that for a normal distribution function, as used above, thecondition α∞1/√{square root over (β)} is required by the normalizationcondition.

Using equation (5) and the arguments above, the probability densityfunction of a zero mean normal distribution for the exemplary cases ofthe first (equation 6), second (equation 7), and fifth (equation 8)derivatives respectively can be derived to be as follows:

$\begin{matrix}\begin{matrix}{\frac{{N\left( {a,\sigma_{0}^{2}} \right)}}{} = {\frac{1}{\sqrt{1^{2} + 1^{2}}}{N\left( {0,\left( \sqrt{2\sigma_{0}} \right)^{2}} \right)}}} \\{= {\frac{1}{\sqrt{2}}{N\left( {0,\left( \sqrt{2\sigma_{0}} \right)^{2}} \right)}}}\end{matrix} & (6) \\\begin{matrix}{\frac{^{2}{N\left( {0,\sigma_{0}^{2}} \right)}}{^{2}} = {\frac{1}{\sqrt{1^{2} + 2^{2} + 1^{2}}}{N\left( {0,\left\lbrack {\left( \sqrt{2^{2} + 1^{2} + 1^{2}} \right)\sigma_{0}} \right\rbrack^{2}} \right)}}} \\{= {\frac{1}{\sqrt{6}}{N\left( {0,\left( {\sqrt{6}\sigma_{0}} \right)^{2}} \right)}}}\end{matrix} & (7) \\\begin{matrix}{\frac{^{5}{N\left( {0,\sigma_{0}^{2}} \right)}}{^{5}} = \frac{1}{\sqrt{1^{2} + 5^{2} + 10^{2} + 10^{2} + 5^{2} + 1^{2}}}} \\{{N\left( {0,\left( {\sqrt{252}\sigma_{0}} \right)^{2}} \right)}} \\{= {\frac{1}{\sqrt{252}}{N\left( {0,\left( {\sqrt{252}\sigma_{0}} \right)^{2}} \right)}}}\end{matrix} & (8)\end{matrix}$

This was indeed verified by numerical simulations where a normaldistributed random set of 200K elements was generated (FIGS. 7 and 8),where FIG. 7 illustrates a simulation of the first derivative (m=1) of a200K normal distributed, N(0,σ₀ ²=1), noise signal (only partiallyshown) with the corresponding deduced histogram (fitted to a Gaussianand arbitrarily scaled). Also shown is the original density functionused to generate the noise signal. As expected, σ=√{square root over(2)}σ₀. Additionally, FIG. 8 illustrates a simulation of the Fifthderivative (m=5) of a normal distributed, noise signal (see also FIG.7). As expected, or σ=√{square root over (252)}σ₀.

In relation to the above, it should be clear that the histograms of theresultant vectors were then taken and are shown to have Gaussian shapeswith variance values compatible with the above results.

Following the above theoretical considerations, it can obviously beconcluded that expressions (3) and (4) can be used in the transformationstep 7 above, while the simplified expression (5) can be used when thedistribution is Gaussian.

To demonstrate one of the proposed motivations for the use of ahigh-order numerical derivative of a stochastic signal, we now refer tothe derivation of the noise-level of an experimental output, wherenoise, either due to experimental set-up or due to the process itself(or due to both), is added to the signal. It is the aim of the followingto demonstrate how to extract a simulated noise component such that thesimulated noise is statistically identical to the noise part in theoriginal experimental signal.

For simplicity we assume that the arbitrary noisy, raw signal can berepresented by an arbitrary smooth and continuous signal contaminated bynoise wherein S=P+N, N being the noise that is added to the pure signalP. Let us further assume that within the interval of validity of P, onecan approximate P (for instance, in the Least Mean Square sense) by anm-degree polynomial function that may belong to a complete orthogonalpolynomial basis. This can be proven to be possible for any bounded,smoothed and continuous function P (see for example the classical proofby K. Weierstrass, Mathematische Werke, Bd. III, Berlin 1903, pp. 1-17,and can also be found in most textbooks on Functional Analysis), but maybe of practical use only when the interval is not too long, as comparedto the structure of the signal, and for a relatively low polynomialdegree.

Assuming the above, it turns out that

$\; {{\frac{^{m + 1}S}{k^{m + 1}} = \frac{^{m + 1}N}{k^{m + 1}}},}$

as the m'th derivative of P, under the above assumptions, is constantand thus vanishes for higher orders. For most experimental data, m wouldnot exceed 5 (see the above example). However the present approach holdsfor any arbitrarily higher order.

Now, if the characteristics of the statistical properties of thehigh-order derivative of the original noise

$\; {\frac{\partial^{m + 1}}{\partial ^{m + 1}}(N)}$

is known, i.e. the probability density function that statisticallydescribes the initial noise subject to high-order numerical derivative,in terms of the parameters (assumed to be unknown) of the statisticalnature of the noise (assumed to be known), one can obtain the specificparameters of the original noise and thus deduce the noise-level in theoriginal signal S.

FIG. 6 illustrates how the method of the present invention can be usedin conjunction with a Kalman (or Extended Kalman) filter. The Kalmanfilter 11 receives at its input the raw signal S, which, as said, is acombination of a pure signal P and of the noise component N. For alinear system, when the distribution of the noise is Gaussian, and giventhe variance σ² ₍₀₎ of the noise distribution, a Kalman filter canprovide at its output a best estimation (optimal) of the pure signal P.In the system of FIG. 6, the raw signal S is provided in parallel toboth the Kalman filter 11 and to the input of the Noise Estimation Unit12, which operates according to the method of the present invention, ormore particularly, according to the method as disclosed in steps 1-7above. The Noise Estimation Unit therefore analyzes the raw signal Saccording to the method of the invention, and provides in real-time tothe Kalman filter over line 13 the variance σ² ₍₀₎ of the noisecomponent N. The variance σ² ₍₀₎ of the noise component is one of thefew parameters of the initial information that the Kalman filterrequires in order to output the estimated pure signal P over line 14.Moreover, the Noise Estimation Unit can output over line 15 both thetype of the probability density function, and the value of the varianceσ² ₍₀₎ to any other component that may require, or use this data.

It should be noted that the exemplary system of FIG. 6 can operateessentially with most types of filters. In that case, another type offilter replaces the Kalman filter 11 of FIG. 6. This feature can beobtained in view of the fact that the Noise Estimation Unit 12 of thepresent invention can operate essentially with most types of probabilitydensity functions, and moreover, the unknown type of the probabilitydensity function, as well as its variance σ² ₍₀₎ can be determined andoutputted by the Noise Estimation Unit 12 of the present invention.

While some embodiments of the invention have been described by way ofillustration, it will be apparent that the invention can be put intopractice with many modifications, variations and adaptations, and withthe use of numerous equivalents or alternative solutions that are withinthe scope of persons skilled in the art, without departing from thespirit of the invention or exceeding the scope of the claims.

1-14. (canceled)
 15. A method for an electronic noise estimation unit tocharacterize noise in a signal, the method comprising: numericallydifferentiating the signal within a window at least m number of times toobtain an m-order differentiated signal; providing a histogram having adistribution that approximates the m-order differentiated signal;providing a probability density function type that approximates thedistribution of the histogram; and outputting the probability densityfunction type.
 16. The method of claim 15, wherein the probabilitydensity function type that approximates the distribution of thehistogram is selected from a plurality of probability density functiontypes.
 17. The method of claim 15, wherein the distribution of thehistogram best fits the m-order differentiated signal and theprobability density function type best fits the distribution of thehistogram.
 18. A method for an electronic noise estimation unit tocharacterize noise in a signal, the method comprising: numericallydifferentiating the signal within a window at least m number of times toobtain an m-order differentiated signal; providing a histogram having adistribution that approximates the m-order differentiated signal;providing a probability density function type that approximates thedistribution of the histogram; determining a variance of the histogram;determining a zero-order variance using the variance of the histogramand the probability density function type; and outputting at least oneof the probability density function type or the zero-order variance. 19.The method of claim 18, wherein the signal comprises a noise componentand a raw signal component.
 20. The method of claim 19, wherein thevariance of the histogram is the m-order variance of the noise componentfrom the signal.
 21. The method of claim 19, wherein the zero-ordervariance is a variance of the probability density function of the noisecomponent from the signal.
 22. The method of claim 18, wherein thewindow of the signal is recorded.
 23. The method of claim 18, whereinthe probability density function type that approximates the distributionof the histogram is selected from a plurality of probability densityfunction types.
 24. The method of claim 18, wherein the distribution ofthe histogram best fits the m-order differentiated signal and theprobability density function type best fits the distribution of thehistogram.
 25. An apparatus for estimating noise in a signal, theapparatus comprising: a differentiation module configured to receive thesignal and numerically differentiate the signal within a range of awindow at least m number of times to obtain an m-order differentiatedsignal; a first module configured to determine a histogram thatapproximates the m-order differentiated signal; a second moduleconfigured to determine a probability density function type thatapproximates the distribution of the histogram; a third moduleconfigured to determine a variance of the histogram; a fourth moduleconfigured to determine a zero-order variance using the variance of thehistogram and the probability density function type; and an interfacemodule configured to output at least one of the zero-order variance orthe probability density function type.
 26. The apparatus according toclaim 25, wherein the apparatus is further configured to determineanother probability density function type and another zero-ordervariance for a noise component of the signal within another range ofanother window.
 27. The apparatus of claim 25, wherein the probabilitydensity function type is determined from a plurality of probabilitydensity function types.
 28. The apparatus of claim 25, wherein thefirst, second, third, and fourth modules are the same module.
 29. Theapparatus of claim 25, wherein the zero-order variance estimates avariance of the probability density function type of a noise componentfrom the signal.
 30. A system for estimating a pure signal component ofa raw signal that includes the pure signal component and a noise signalcomponent, the system comprising: an electronic noise estimation unitincluding: a differentiation module configured to receive a raw signaland numerically differentiate the raw signal within a range of a windowat least m number of times to obtain an m-order differentiated signal; afirst module configured to determine a histogram that approximates them-order differentiated signal; a second module configured to determine aprobability density function type that approximates the distribution ofthe histogram; a third module configured to determine a variance of thehistogram; and a fourth module configured to determine a zero-ordervariance using the variance of the histogram and the probability densityfunction type; and a filter coupled to the noise estimation unit,wherein the filter is configured to receive the zero-order variance andthe raw signal and to output the estimated pure signal component of theraw signal.
 31. The system of claim 30, wherein the filter is anadaptive filter.
 32. The system of claim 30, wherein the filter is aKalman Filter.
 33. The system of claim 30, wherein the electronic noiseestimation unit is configured to determine a zero-order variance on theraw signal in real time.